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Propulsion ground test facilities face the daily challenge of scheduling multiple customers 
into limited facility space and successfully completing their propulsion test projects. Over 
the last decade NASA’s propulsion test facilities have performed hundreds of tests, collected 
thousands of seconds of test data, and exceeded the capabilities of numerous test facility and 
test article components. A logistic regression mathematical modeling technique has been 
developed to predict the probability of successfully completing a rocket propulsion test. A 
logistic regression model is a mathematical modeling approach that can be used to describe 
the relationship of several independent predictor variables Xj, X 2 ,.., X k to a binary or 
dichotomous dependent variable Y, where Y can only be one of two possible outcomes, in 
this case Success or Failure of accomplishing a full duration test. The use of logistic 
regression modeling is not new; however, modeling propulsion ground test facilities using 
logistic regression is both a new and unique application of the statistical technique. Results 
from this type of model provide project managers with insight and confidence into the 
effectiveness of rocket propulsion ground testing. 
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I. Introduction 

Today’s increased emphasis on streamlining the propulsion system development process is increasing the pressure 
on propulsion system test centers to develop innovative approaches and increase utilization of current test facility 
assets. The role of rocket engine propulsion ground test facilities covers a wide landscape from dedicated single use 
rocket engine test stands supporting space shuttle and commercial launch systems thru multi-cell, multi-purpose 
rocket propulsion research and development test facilities. Full scale rocket engine test stands are normally several 
stories tall and designed to withstand several hundred thousand pounds of thrust, while component test cells are 
approximately twenty to thirty feet wide by twenty to thirty feet deep, with a structural blast wall separating the 
support equipment, instrumentation connections, power and safety equipment. Rocket propulsion ground test 
facilities have the capability to deliver a diverse selection of fuel and oxidizer propellants at low, medium, high and 
ultra high pressures with flows ranging from a few pounds per second up to a two thousand pounds per second. 
Additionally, propulsion test facilities are generally equipped to supply a wide range of high and low pressure gases, 
hydraulics, deluge water, potable water, high-speed video, low speed video and 35mm photography. See Figure 1 
for a schematic of a generic rocket propulsion test facility configuration. 
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Figure 1. Generic Rocket Propulsion Test Facility 

Test facility data acquisition systems on the whole have tremendous capacities and are the state-of-the-art in number 
of data channels and data storage capability. Test facility control systems are set up to provide a real-time graphical 
display of operations in progress, which can be monitored by test personnel in the test control center and in some 
cases at remote sites. Typical propellants found in rocket propulsion test facilities are liquid oxygen, liquid 
hydrogen, hydrogen peroxide and kerosene. Typical types of rocket propulsion tests are turbo-pump testing, injector 
testing, combustion testing and single engine testing. 

In general a successful test is defined as a test that reaches full duration and the test article remains structurally 
sound. Off nominal performance by the test article is not considered a test failure if the test is not prematurely ended 
due to a violation of hardware constraint. A test anomaly can be described as a failure of a component in the facility 
or on the test article, such as an instrument. Anomalies are usually investigated and/or repaired prior to the next test. 
A test failure is defined as the termination of a test prior to reaching full duration due to any test termination event. 
These events could include automated facility or test article redline violations or manual observer-initiated cutoff. 

Over the last decade rocket propulsion ground test facilities have performed a multitude of tests, collected reams of 
test data, and exceeded the operating capabilities of numerous components. Many valuable technical lessons have 
been learned, better propulsion test techniques developed, and various initiatives have been implemented to provide 
better propulsion test facility preventive maintenance to improve the quality of goods and parts used in the 
propulsion test facilities. Even with these advances, unexpected failures during testing still occur regularly due to 
the harsh environment in which the propulsion test facilities and propulsion systems operate, eroding the ability of 
the test facility to meet test goals, on schedule and on budget. 
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II. Predicting Successful Rocket Propulsion Tests 

Most propulsion system testing is undertaken to raise the level of technology maturity or expand the operating 
envelope. That lack of technology maturity in the propulsion test articles, combined with pushing the test facilities 
capabilities to their limits, tends to lead to an increase in facility breakdowns and unsuccessful tests. In today’s 
environment of higher material and labor costs, chronic underfunding, short turnaround times and ill-defined 
customer requirements, knowledge is critical. Schedule estimates are one of the primary issues confronting rocket 
propulsion test facilities. Turnaround time between tests can be longer than anticipated. Facility failures during 
testing can be higher than expected. Propulsion systems customers no longer are just satisfied with successfully 
completing the testing; they want the testing complete within the original schedule estimate and on budget. The 
purpose of this paper is to provide an increased understanding of the probability and consequences of unexpected 
failures that occur during the testing phase of a rocket propulsion system development project. The research and 
model development provides information that project managers and test directors can use during the planning phase 
of a project to assess the potential schedule impacts caused by unexpected testing failures, thus allowing the project 
manager or test director to minimize the risk, minimize the cost, and maximize the probability of test success. 

Most of the propulsion testing modeling efforts to date concentrated on the entire rocket propulsion test project 
lifecycle. Multiple attempts at modeling the complete propulsion test project lifecycle have been undertaken 1 ' 2 . 
While technically accurate, each of the models suffers from incomplete and inconsistent data sets on which to base 
the models. Several attempts were made to improve the models, including the development of a tool for predicting 
test cell throughput capacity using queuing theory 3 , in which the testing process was modeled as a single server 
system (M/M/1) queuing process. Unfortunately, the model error rate of 17.2% was still quite high. While modeling 
the life cycle of a test project as it moves through the test facility is technically feasible, quality sources of data for 
each phase of the test project are not readily available, which has left the test facility operators unable to provide the 
customer accurate insight into the propulsion test facilities’ ability to meet total project schedule milestones or 
individual test milestones. Sufficiently validated data does not exist to accurately model the entire rocket propulsion 
test project life cycle. However, a large set of quality data does exist covering the actual testing phase for propulsion 
test facilities. By concentrating on predicting the ability of a test facility to successfully perform tests in a timely 
manner using the available data, the inaccuracy and inconstancies of trying to model the entire propulsion test 
project lifecycle with inadequate data resources is eliminated, thus, providing the customer with insight into test 
facilities’ ability to meet schedule milestones in the testing phase of the rocket propulsion test project lifecycle. 

The goal of this paper is to provide decision makers with a technique to use when collecting relevant information for 
developing insight into a rocket propulsion test facilities’ ability to meet test schedule milestones, providing the 
customer with a realistic test schedule with a high confidence level. The method for performing these calculations 
calls for the fitting of a logistic regression model to available test data. A test facility with sufficient operating 
history to provide a large data set was needed. Stennis Space Center’s E-complex meets that criterion. Stennis’ E- 
complex has performed hundreds of rocket propulsion tests over the last decade. The paper will concentrate on 
Stennis’ E-complex test facilities large collection of recent data. 


III. Rocket Propulsion Facility Test Data 

Why use Stennis to develop the model? Stennis Space Center, located in south Mississippi, holds the responsibility 
for all NASA’s rocket propulsion test activities 4 . The center’s management responsibilities include NASA’s rocket 
propulsion test assets and activities in Mississippi, Alabama, Ohio and New Mexico. SSC provides the leadership 
and stewardship roles in conducting rocket propulsion testing. Stennis tests and flight certifies rocket propulsion 
systems for current and future space vehicles and provides test services for both government and commercial 
customers. Stennis’ propulsion test areas consist of the A, B and E test complexes. The A & B test complexes 
provide large rocket engine test services, while E-complex provides small rocket engine and component test 
services. Stennis Space Center’s E complex consists of three test facilities with seven test cells. Over the years, 
Stennis has amassed large quantities of data that reflect how and what it takes to operate propulsion test facilities. 
Safe and affordable space transportation is a top priority for NASA. Stennis Space Center continues to seek ways to 
reduce the risk associated with developing and testing technologies needed to access space safely and within budget. 
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Specifically Stennis’ E-complex has been chosen for this research effort due to the large volume of recent test 
activity. Hundreds of tests have been performed with thousands of seconds of data collected and a multitude of 
hardware issues over the last decade. The E complex consists of El shown in Figure 2, E2 shown in Figure 3 and 
E3 shown in Figure 4. Initial construction of the $40 million NASA complex was begun on January 18, 1989. 
Originally known as the Component Test Facility (CTF), the E-complex propulsion ground test facility covers more 
than 20 acres in territory and was originally scheduled to be operational by mid-summer of 1991. Designed to test 
high-fidelity turbo-pump assemblies for the proposed heavy-lift Advanced Launch System (ALS) being jointly 
developed by NASA and the US Air Force, construction of the new test facility was to take two years. The ALS 
program eventually became the New Launch System (NLS) program, which unfortunately was cancelled. In late 
1996 the Reusable Launch Vehicle (RLV) program was initiated. With this announcement, came $45 million in the 
FY 97 budget to finish CTF, which was renamed E complex. The role of the Stennis Space Center and E complex 
in this new program was expanded to provide a facility for component level testing of the RLV engine valves, 
thrusters, and turbo-pumps. Additionally, the test facility was reconfigured to perform horizontal firings of rocket 
engines. While the RLV program has faded into history, E complex’s role as a preeminent propulsion ground test 
site endures as we continue to work toward the NASA’s Vision for Space Exploration. 



Figure 2. El Test Facility. 
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Figure 3. E2 Test Facility. 



Figure 4. E3 Test Facility. 
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IV. Development of the Logistic Regression Model 

Rocket propulsion test facilities need to instill a high level of confidence in their customers by providing insight into 
the propulsion test facilities ability to meet the test schedule milestones. To provide this insight a logistic regression 
model was developed to calculate the probability of successfully completing a given test. 

A Logistic Regression model is a mathematical modeling approach that can be used to describe the relationship of 

several independent predictor variables (covariates) XI, X2, Xk to a binary or dichotomous dependent variable 

Y, where Y can on be one of two possible outcomes 5 , in this case Success or Failure. 

Conditional Probability 

P(Y = 1 1 x) = 7t(x) (1) 

Logit of the Multiple Logistic Regression Model 

g(x) = /?„ + p x X x + p 2 X 2 + ...+ fi p X p (2) 


Logistic Regression Model: 

Jt(x)= e 8(x) /(\ + e 8(x) ) 


( 3 ) 


The use of logistic regression modeling was originally developed for epidemiologic research by Lemshow & 
Homer 6 and Kelsey, Thompson, & Evans 7 . The method is now commonly employed in many fields including 
biomedical research, business and finance, criminology, ecology, health policy, linguistics and wildlife biology. 

As indicated in the previous paragraph the use of logistic regression is not new. However, logistic regression 
analysis has not been applied to rocket propulsion testing. Modeling propulsion test facilities using logistic 
regression is a new and unique application of the statistical technique. Results developed from models using this 
statistical technique will provide project managers with insight and a higher level of confidence in a propulsion test 
facilities’ ability to meet test schedules. 

The selection of independent variables for use in a Logistic Regression Model is the key component in the 
development of a useful tool for predicting the probability of successfully testing propulsion systems. Independent 
variables to consider include pressure, temperature, fuel type, oxidizer type, test duration, test article TRL level, 
instrumentation count and complexity, type of test article, and many others. If we were to look at the probability of 
completing multiple tests within a given period, independent variables such as number of work shifts, test 
turnaround duration, and number of predicted weather delays also become important variables to consider. 

The following example uses a randomly selected set of 300 test data points collected from facility activation tests or 
propulsion hotfire tests conducted at Stennis Space Center over the last decade. Maximum facility propellant 
pressure and test duration were chosen as the independent variables. The dependent variable is the success or failure 
of the test. 

Schedule durations for designing or constructing the propulsion test facilities are not to be considered. Test article 
testing requirements are considered to be mature and any changes to the facility or test article are considered to be 
minor. The definition of a successful test is defined as the test reaches full duration as defined in the test request and 
the test facility and test article retain their structural integrity. Performance of the test article beyond it maintaining 
structural integrity was not considered in the model. 

Test Durations range from 1 to 700 seconds. Max Facility Test Pressure ranges from 15 psi to 8500 psi. Test 
Duration was grouped into one of twelve bins and Max Test Pressure was grouped into one of seven bins, as seen in 
table 1. A test’s outcome is labeled as “S = 1” for a successful test and a test failure is labeled as “F = 0”. A subset 
of the data can be seen in table 2. 
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Test Duration Bin 

Test Duration Bin Range 

Max Facility Test Pressure 
Bin 

Max Facility Test Pressure 
Bin Range 

1 

1 to 15 sec 

1 

15 to 500 psi 

2 

16 to 30 sec 

2 

501 to 1500 psi 

3 

31 to 45 sec 

3 

1501 to 3000 psi 

4 

46 to 60 sec 

4 

3001 to 4500 psi 

5 

61 to 75 sec 

5 

4501 to 6000 psi 

6 

76 to 90 sec 

6 

6001 to 7500 psi 

7 

91 to 120 sec 

7 

Greater than 7500 psi 

8 

121 to 180 sec 



9 

181 to 225 sec 



10 

225 to 300 sec 



11 

301 sec to 400 sec 



12 

Greater than 400 sec 




Table 1: Test Duration Bins and Maximum Facility Test Pressure Bins 


Test Number 

Success 01 - 
Failure 

Test Duration 

Maximum 
Facility Test 
Pressure 

S = 1 or F = 0 

TDUR 

Bin 

MAXP 

Bin 

1 

S 

2.82 

1750.00 

1 

2 

1 

2 

S 

3.00 

716.00 

1 

2 

1 

3 

s 

4.00 

850.00 

1 

2 

1 

4 

s 

5.00 

70.00 

1 

1 

1 

5 

F 

5.00 

113.00 

0 

1 

1 

6 

s 

5.00 

200.00 

1 

1 

1 

7 

s 

5.00 

200.00 

1 

1 

1 

8 

s 

5.00 

200.00 

1 

1 

1 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

* 

296 

s 

700.00 

400.00 

1 

2 

12 

297 

s 

700.00 

400.00 

1 

2 

12 

298 

s 

700.00 

400.00 

1 

2 

12 

299 

s 

700.00 

400.00 

1 

2 

12 

300 

s 

1150.00 

400.00 

1 

2 

12 


Table 2: An example of the rocket propulsion test series data 
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V. Binary Logistic Regression: S-F versus TA Secs 


Logistic Regression analysis was performed using Test Duration and Max Test Pressure as the independent predictor 
variables and the Test Success or Failure as the dichotomous dependent variable. Solving the logistic regression 
yielded the following results 8 . TDUR represents Test Duration and MAXP represents Maximum Facility Test 
Pressure. 


Response Information 


Variable Value 
S=1/F=0 1 

0 

Total 


Count 

258 (Event) 
42 
300 


Logistic Regression Table 


Odds 95% Cl 


Predictor 

Constant 

Coef 

1.12574 

SE Coef 
0.416458 

Z 

2.70 

Ratio 

Lower 

Upper 

TDUR 

0.16277 

0.078155 

2.08 

1.18 

1.01 

1.37 

MAXP 

0.10530 

0.191383 

0.55 

1.11 

0.76 

1.62 


Log-Likelihood = -1 18.086 

The logistic regression model can now be defined as: 
Logit of the Multiple Logistic Regression Model 


g(x) = 1 . 12574 + 0. \6211TDUR{x) + 0. 10530M4XP(x) (4) 


Logistic Regression Model: 


\ 1.12574+0. 1627777) t/t?(x)+0.10530M4ZP(;c 

Ji 1 A I — id 


'/(! + «' 


12574+0 . 16217 TDUR(x)+0A0530MAXP(x) 


(5) 


Furthermore from the model it can be concluded that Test Duration and Max Facility Test Pressure are significant 
predictors of Test Success/Failure. This can be further visualized in Table 3. 


Group 

Value 


1 

2 

3 

4 

5 

6 

7 

8 

Total 

1 












Observed 

27 

64 

33 

31 

31 

29 

31 

12 

258 


Expected 

28.8 

58.9 

38.5 

31.0 

28.2 

29.1 

32.0 

11.5 


0 












Observed 

9 

8 

13 

5 

1 

3 

3 

0 

42 


Expected 

7.2 

13.1 

7.5 

5.0 

3.8 

2.9 

2.0 

0.5 


Total 


36 

72 

46 

36 

32 

32 

34 

12 

300 


Table 3. Table of Observed and Expected Frequencies. 
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A plot of the Logistic Regression Model derived in Equation (5) can be seen in Figure 5. The plot suggests that in 
general as the maximum facility pressure increases the likelihood of a successful test increases. The plot also, 
suggests as test duration increases the likelihood of a successful test increases. 


Probability of Successfully Testing vs. Duration of Test 
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Figure 5. Probability of Successfully Testing vs. Duration of Test Given a Facility Pressure 


The highest probability of a successful test (0.978) translates into a pressure of 7500 to 9000 psi with a test duration 
of over 300 seconds. The lowest probability of successful test (0.802) translates into a pressure less than 500 psi 
with a test duration up to 15 seconds. Overall, a successful test is more likely than an unsuccessful test. The TDUR 
odds ratio (OR) point estimate, OR = 1.18, indicates that a successful rocket propulsion test is 1.18 times more 
likely than an unsuccessfully test as the test duration increases. The 95% confidence interval suggests that 
successful tests could be as little as 1.01 times or as much as 1.37 times more likely than an unsuccessful test as the 
test duration increases. Additionally, the MAXP odds ratio (OR) point estimate, OR = 1.11, indicates that the 
likelihood of a successful rocket propulsion test increase as the maximum facility pressure increases. The 95% 
confidence interval suggests that the likelihood of a successful test could be as little as 0.76 times or as much as 1.62 
times the original estimate, as the maximum facility test pressure increases. The results provide evidence to support 
the theory that most test failures occur early in a test series when the test article and facility are still searching for 
their respective sweet spots. The high probability of success as duration increases reflects the confidence that the 
facility and test article are ready to test for longer duration when the risk of the unknown has been reduced. As with 
most regression analysis, the larger the sample set, the better the fit of the model. As more data becomes available 
the accuracy of the Logistic Regression model will continue to improve. 
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VI. Conclusion 

Propulsion test facilities face the daily challenge of scheduling multiple customers into limited test facility space. 
Test schedules provide customers information and knowledge identifying the propulsion test facilities level of 
confidence in estimating work. Attempts made in the past to model the complete lifecycle of a test project have met 
with little success. Each of the attempts suffered from incomplete or inconsistent data on which to base the models. 
By focusing on the actual test phase of the project rather than the formulation, design or construction phases of the 
project, the quality and quantity of available data increases dramatically. 

Using a random sample of the data collected over the last decade a logistic regression model was developed to 
calculate the probability of successfully completing a rocket propulsion test. A more accurate depiction of specific 
propulsion test facilities capability can be achieved by sorting the data to only use data collected from that facility. If 
the data set was large enough, the data could be further refined to use only data taken from a specific type of testing. 


This initial study presented on the previous pages was undertaken to prove the feasibility of using logistic regression 
to model rocket propulsion testing. The successful development of the example model clears the way for more 
complex logistic regression models to be developed in this area. From a customer perspective the results build 
confidence in the propulsion test facilities’ ability to meet scheduled test milestones by providing an estimate on 
how successful the testing will be from the facilities’ point of view. From a propulsion test facility’s perspective 
the results impart insight into whether the test facilities are being over-extended and how it impacts the customers. 
This same logistic regression technique can be used with success at any other NASA or industry propulsion test 
facilities where large sets of test data can be found. 
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